Accelerating Cooperative Planning for Automated Vehicles with Learned Heuristics and Monte Carlo Tree Search

Efficient driving in urban traffic scenarios requires foresight. The observation of other traffic participants and the inference of their possible next actions depending on the own action is considered cooperative prediction and planning. Humans are well equipped with the capability to predict the actions of multiple interacting traffic participants and plan accordingly, without the need to directly communicate with others. Prior work has shown that it is possible to achieve effective cooperative planning without the need for explicit communication. However, the search space for cooperative plans is so large that most of the computational budget is spent on exploring the search space in unpromising regions that are far away from the solution. To accelerate the planning process, we combined learned heuristics with a cooperative planning method to guide the search towards regions with promising actions, yielding better solutions at lower computational costs

Decentralized Cooperative Planning for Automated Vehicles with Continuous Monte Carlo Tree Search

Urban traffic scenarios often require a high degree of cooperation between traffic participants to ensure safety and efficiency. Observing the behavior of others, humans infer whether or not others are cooperating. This work aims to extend the capabilities of automated vehicles, enabling them to cooperate implicitly in heterogeneous environments. Continuous actions allow for arbitrary trajectories and hence are applicable to a much wider class of problems than existing cooperative approaches with discrete action spaces. Based on cooperative modeling of other agents, Monte Carlo Tree Search (MCTS) in conjunction with Decoupled-UCT evaluates the action-values of each agent in a cooperative and decentralized way, respecting the interdependence of actions among traffic participants. The extension to continuous action spaces is addressed by incorporating novel MCTS-specific enhancements for efficient search space exploration. The proposed algorithm is evaluated under different scenarios, showing that the algorithm is able to achieve effective cooperative planning and generate solutions egocentric planning fails to identify

Decentralized Cooperative Planning for Automated Vehicles with Hierarchical Monte Carlo Tree Search

Today's automated vehicles lack the ability to cooperate implicitly with others. This work presents a Monte Carlo Tree Search (MCTS) based approach for decentralized cooperative planning using macro-actions for automated vehicles in heterogeneous environments. Based on cooperative modeling of other agents and Decoupled-UCT (a variant of MCTS), the algorithm evaluates the state-action-values of each agent in a cooperative and decentralized manner, explicitly modeling the interdependence of actions between traffic participants. Macro-actions allow for temporal extension over multiple time steps and increase the effective search depth requiring fewer iterations to plan over longer horizons. Without predefined policies for macro-actions, the algorithm simultaneously learns policies over and within macro-actions. The proposed method is evaluated under several conflict scenarios, showing that the algorithm can achieve effective cooperative planning with learned macro-actions in heterogeneous environments

Root Canal Obturation by Electrochemical Precipitation of Calcium Phosphates

Achieving adequate disinfection and preventing reinfection is the major goal in endodontic treatment. Variation in canal morphology and open porosity of dentine prevents achieving complete disinfection. Questionable biocompatibility of materials as well as a lack of sealing ability questions the usefulness of current obturation methods. With a novel disinfection approach based on the use of boron-doped diamond (BDD) electrodes having shown promising results it was the goal of this series of experiments to investigate the possibility of BDD-mediated in situ forming of a biocompatible obturation material. A combination of calcium phosphate and maleic acid was used as precursor solution while Ion Chromatography Mass Spectrometry (IC-MS), Raman spectroscopy (RAMAN), X-ray diffraction (XRD), energy dispersive X-ray spectroscopy (EDX), scanning electron microscopy (SEM), dye penetration and micro-computed tomography (µCT) were applied for characterizing the precipitate. It was possible to achieve a BDD-mediated precipitation of brushite in a clinically applicable timeframe. However, tight sealing of the canal system based on brushite could not be achieved

Path Planning in Unstructured Environments : A Real-time Hybrid A* Implementation for Fast and Deterministic Path Generation for the KTH Research Concept Vehicle

On the way to fully autonomously driving vehicles a multitude of challenges have to be overcome. One common problem is the navigation of the vehicle from a start pose to a goal pose in an environment that does not provide any specic structure (no preferred ways of movement). Typical examples of such environments are parking lots or construction sites; in these scenarios the vehicle needs to navigate safely around obstacles ideally using the optimal (with regard to a specied parameter) path between the start and the goal pose. The work conducted throughout this master's thesis focuses on the development of a suitable path planning algorithm for the Research Concept Vehicle (RCV) of the Integrated Transport Research Lab (ITRL) at KTH Royal Institute of Technology, in Stockholm, Sweden. The development of the path planner requires more than just the pure algorithm, as the code needs to be tested and respective results evaluated. In addition, the resulting algorithm needs to be wrapped in a way that it can be deployed easily and interfaced with di erent other systems on the research vehicle. Thus the thesis also tries to gives insights into ways of achieving realtime capabilities necessary for experimental testing as well as on how to setup a visualization environment for simulation and debugging

Cooperative Trajectory Planning in Uncertain Environments with Monte Carlo Tree Search and Risk Metrics

Automated vehicles require the ability to cooperate with humans for a smooth integration into today's traffic. While the concept of cooperation is well known, the development of a robust and efficient cooperative trajectory planning method is still a challenge. One aspect of this challenge is the uncertainty surrounding the state of the environment due to limited sensor accuracy. This uncertainty can be represented by a Partially Observable Markov Decision Process. Our work addresses this problem by extending an existing cooperative trajectory planning approach based on Monte Carlo Tree Search for continuous action spaces. It does so by explicitly modeling uncertainties in the form of a root belief state, from which start states for trees are sampled. After the trees have been constructed with Monte Carlo Tree Search, their results are aggregated into return distributions using kernel regression. For the final selection, we apply two risk metrics, namely a Lower Confidence Bound and a Conditional Value at Risk. It can be demonstrated that the integration of risk metrics in the final selection policy consistently outperforms a baseline in uncertain environments, generating considerably safer trajectories

Learning Reward Models for Cooperative Trajectory Planning with Inverse Reinforcement Learning and Monte Carlo Tree Search

Cooperative trajectory planning methods for automated vehicles can solve traffic scenarios that require a high degree of cooperation between traffic participants. However, for cooperative systems to integrate into human-centered traffic, the automated systems must behave human-like so that humans can anticipate the system's decisions. While Reinforcement Learning has made remarkable progress in solving the decision-making part, it is non-trivial to parameterize a reward model that yields predictable actions. This work employs feature-based Maximum Entropy Inverse Reinforcement Learning combined with Monte Carlo Tree Search to learn reward models that maximize the likelihood of recorded multi-agent cooperative expert trajectories. The evaluation demonstrates that the approach can recover a reasonable reward model that mimics the expert and performs similarly to a manually tuned baseline reward model

Root Canal Obturation by Electrochemical Precipitation of Calcium Phosphates

Achieving adequate disinfection and preventing reinfection is the major goal in endodontic treatment. Variation in canal morphology and open porosity of dentine prevents achieving complete disinfection. Questionable biocompatibility of materials as well as a lack of sealing ability questions the usefulness of current obturation methods. With a novel disinfection approach based on the use of boron-doped diamond (BDD) electrodes having shown promising results it was the goal of this series of experiments to investigate the possibility of BDD-mediated in situ forming of a biocompatible obturation material. A combination of calcium phosphate and maleic acid was used as precursor solution while Ion Chromatography Mass Spectrometry (IC-MS), Raman spectroscopy (RAMAN), X-ray diffraction (XRD), energy dispersive X-ray spectroscopy (EDX), scanning electron microscopy (SEM), dye penetration and micro-computed tomography (µCT) were applied for characterizing the precipitate. It was possible to achieve a BDD-mediated precipitation of brushite in a clinically applicable timeframe. However, tight sealing of the canal system based on brushite could not be achieved